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Introduction  by  Jason  Warner,  FMSO 

Since  the  international  community  first  became  engaged  in  the  process  of post-conflict 
reconstruction,  observers  around  the  world  have  been  seeking  effective  ways  to  monitor 
outcomes  of  these  efforts.  Despite  this  long-held  attention,  however,  a  consensus  has  yet  to  be 
reached  on  which  metric  for  evaluation  should  be  used.  As  a  result,  the  process  of  monitoring 
post-conflict  rebuilding  efforts  remains  scattershot  and  un-streamlined. 

To  this  end,  authors  Davids,  Rietjens  and  Soeters  argue  that  the  creation  of  a  unified 
metric  would  have  innumerable  benefits  for  making  future  reconstruction  efforts  more 
effective.  To  lay  the  groundwork  for  such  a  project,  they  turn  their  sights  to  reconstruction 
efforts  in  Afghanistan.  Using  the  Afghan  Country  Stability  Picture  (ACSP)  —  a  database 
of  some  85,000  projects  —  they  seek  to  find  correlative  patterns  between  donor  priorities, 
geographical  locations,  aid  effectiveness  and  tendencies  for  project  completion.  Ultimately, 
they  hope  to  use  information  gleaned  from  their  analysis  of  Afghanistan  for  the  creation  of  a 
tool  to  assess  post-conflict  reconstruction  across  the  world. 

This  paper  admirably  compels  readers  regarding  the  need  to  streamline  the  evaluation 
process  of  international  community  nation-building  efforts.  However,  several  questions 
remain:  in  the  real  world,  how  is  such  a  process  to  be  standardized,  particularly  when  past 
efforts  by  the  international  development  community  to  create  similar  monitoring  criteria 
have  had  only  limited  success?  Similarly,  given  that  resources  devoted  to  reconstruction 
projects  are  already  scarce,  how  does  a  coun  try  or  coalition  devote  a  greater  share  of  each 
organization  s  operating  budget  towards  monitoring  —  especially  when  such  funds  could 
be  used  to  provide  tangible  necessities  such  as  food,  healthcare  and  education?  Finally,  the 
question  of  extrapolation  lingers.  Is  it  a  risk  to  assume  that  the  same  assessment  metric  can 
be  applied  universally  across  world  regions,  despite  wildly  divergent  ground-level  realities? 
Would  this  approach  obscure  the  reality  more  than  it  would  reveal  it? 

These  are  tough  questions  to  a  challenging  endeavor  undertaken  by  Davids,  Rietjens  and 
Soeters,  whose  substantive  research  has,  at  a  minimum,  improved  upon  the  evaluation  of 
post-conflict  reconstruction. 
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Abstract 

ation  building  and  its  supporting  policy  development  should  no  longer  occur  without  a  careful 
cost-benefit  analysis.  This  article  attempts  to  measure  the  progress  of  the  Afghan  reconstruction 
process,  in  which  the  militaries  of  42  countries  and  many  different  international  and  local  humanitarian 
organizations  are  currently  engaged.  It  presents  results  from  the  Afghan  Country  Stability  Picture 
(ACSP),  a  database  containing  detailed  information  on  approximately  85,000  projects  (related  to  such 

sectors  as  water,  energy,  education,  and  governance)  in  Afghanistan  between 
2002  and  2008.  Using  this  quantitative  database,  we  reveal  descriptive  findings 
and  correlative  relations  between  project  variables  such  as  timeframe,  location, 
costs,  donor  characteristics,  the  security  situation,  and  learning  experiences.  We 
argue  that  the  correlations  between  these  variables,  as  seen  in  the  ACSP,  may 
be  suitable  to  develop  a  wider  framework  capable  of  gauging  the  prospects  for 
both  stability  and  prosperity  in  other  post-conflict  situations. 
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org/licenses/by-sa/3.0/)] 


1.  Introduction 

The  international  community  is  frequently  called  upon  to  stabilize  countries 
and  regions  affected  by  conflict.  In  these  operations,  military  and  civilian 
workers  operate  together  in  states  where  local  administrations  and  security 
organizations  often  perform  inadequately  and  basic  public  services  are 
lacking.  Whereas  peacekeeping  in  the  Cold  War  era  was  typically  limited  to 
monitoring  ceasefires  between  two  parties  and  manning  buffer-zones,  today 
Western  nations  have  developed  an  interest  in  achieving  more  ambitious  goals, 
including  -  most  importantly  for  this  study  -  the  re-building  of  nations  in 
post-conflict  situations  (Kang  and  Meemik,  2004;  Ghani  and  Lockhart,  2008). 
Therein,  Western  nations’  goals  include  aiding  in  stabilization,  providing 
humanitarian  aid,  facilitating  post-war  reconstruction,  encouraging  economic 
and  social  rehabilitation,  leading  security  sector  reform  and  promoting 
democratization. 

Although  nation  building  and  reconstruction  attract  a  great  deal  of  attention 
these  days,  these  processes  are  decidedly  not  new.  Various  interventions  by 
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the  United  States  and  certain  European  powers  -  in  places  such  as  South  Korea,  Japan,  Haiti  and  East 
Timor  -  have  preceded  the  current  activities  in  Iraq  and  Afghanistan  (e.g.  Fukuyama,  2006).  In  the  latter, 
previous  attempts  to  modernize  the  country’s  infrastructure  and  security  sector  testify  that  the  ambition  to 
develop  a  nation  from  outside  is  not  a  new  phenomenon  (Cullather,  2002). 

Throughout  the  history  of  nation  building,  participants  have  recognized  the  undeniable  importance  of 
assessment  and  monitoring.  In  an  attempt  to  go  beyond  the  formulation  of  simply  “visions  of  the  future,” 
the  assessment  of  post-conflict  nation  building  is  characterized  by  its  focus  on  the  actual  implementation 
of  projects  and  the  delivery  of  collective  goods  and  services  (Ghani  and  Lockhart,  2008).  By  assessing 
such  progress  it  may  be  possible  to  improve  the  coordination,  communication  and  accountability  of  the 
activities  of  the  contributing  organizations  and  countries  (Diehl,  2008;  Fast  and  Neufeldt,  2005). 

To  this  end,  observers  have  given  widespread  attention  to  performance  measurement  in  conflict 
literature  (e.g.  Chauvet  &  Collier,  2008;  Donini,  2007;  Freeman,  2007;  Rietjens  &  Bollen,  2007). 
Throughout  these  various  perspectives,  however,  we  contend  that  there  is  no  singular,  widely-used 
framework  to  interpret  the  progress  achieved  by  organizations  engaged  in  nation  building  (e.g.,  Rietjens 
and  Bollen,  2008).  Nor  is  there  a  solid  set  of  numbers  for  measurement  to  form  an  understanding  of 
the  raw  data  (Clancy  and  Crosset,  2007).  Many  policy  evaluations  in  this  field  are  built  on  qualitative 
descriptions  and  assessments  of  the  developments  under  scrutiny  (e.g.,  Tondini,  2007).  Of  course,  these 
evaluations  are  worthwhile,  but  such  approaches  lack  the  insights  and  reliability  that  go  beyond  the 
inherent  limitations  of  the  individual  evaluator. 

This  article  represents  an  attempt  to  measure  nation  building  progress  and  reconstruction  in  Afghanistan 
based  on  the  analysis  of  quantitative  data.  This  data  is  taken  from  the  Afghanistan  Country  Stability 
Picture  (ACSP),  a  database  containing  information  on  reconstruction  and  development  projects  from 
all  over  the  country.  The  result  is  a  dataset  addressing  approximately  85,000  projects  undertaken  in 
Afghanistan  between  2002  and  2008.  For  each  project,  ACSP  records  several  characteristics  including  the 
start  date,  cost,  location  and  the  pillar  of  the  Afghanistan  National  Development  Strategy  (ANDS)  under 
which  the  project  is  categorized  (for  example,  governance,  infrastructure,  or  health  and  nutrition). 

Using  the  ACSP  to  measure  the  progress  and  performance  of  organizations  in  Afghan  can  be 
advantageous.  The  database  is  widely  supported  by  both  military  and  civilian  actors  in  the  country,  which 
minimizes  potential  friction  over  the  figures  that  could  otherwise  arise  between  the  two  entities.  Updating 
ACSP  requires  few  resources  and  therefore  minimizes  the  need  for  extra  bureaucracy.  Finally,  the  project 
records  in  ACSP  are  relatively  simple,  objective,  and  are  presented  as  quantitative  data  that  give  little 
space  for  discussion.  Despite  these  advantages,  ACSP’s  limitation  is  that  it  focuses  on  efforts  and  outputs 
of  projects,  rather  than  collective  societal  outcomes  such  as  the  improvement  of  the  health  situation  of  the 
Afghan  population  in  a  particular  province.  Therefore,  while  ACSP  does  contribute  in  substantive  ways 
to  understanding  the  efficacy  of  post-conflict  reconstruction  efforts  in  Afghanistan,  by  no  means  does  it 
represent  a  flawless  dataset. 

In  sum,  this  article  provides  new  insights  and  understandings  of  reconstruction  activities  in  Afghanistan 
through  quantitative  analysis,  and  contributes  to  the  development  and  use  of  progress  and  performance- 
based  measurements  in  nation  building  and  reconstruction  more  broadly.  The  next  section  provides  an 
overview  of  our  understanding  of  progress  and  performance  measurement  in  the  public  sector  in  general 
and  in  nation  building  and  reconstruction  projects  in  particular.  Section  three  outlines  the  methodology  of 
our  analysis,  while  the  fourth  section  provides  descriptive  and  explanatory  analyses  of  the  ACSP  dataset. 
The  article  ends  with  conclusions  about  future  ways  of  improving  both  the  data  and  its  use  in  policy 
evaluation. 
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2.  Measuring  Progress  and  Performance  in  Nation  building 

Foremost,  it  is  important  to  understand  the  notion  of  “management  accounting,”  or  the  process  of 
monitoring  and  assessing  projects  and  their  application  to  reconstruction  efforts.  Bisbe  and  Otley  (2004) 
refer  to  such  management  accounting  systems  simply  as  a  set  of  “procedures  and  processes  that  managers 
and  other  organizational  participants  use  in  order  to  help  ensure  the  achievement  of  their  goals  and 
the  goals  of  their  organizations.”  To  date,  management  accounting  and  control  systems  have  received 
insufficient  attention  in  the  context  of  defense  organizations  and  nation  building  activities  (i.e.  Gronlund 
and  Catasus,  2005;  Lambert,  2002).  This  is  regrettable,  because  lessons  from  this  field  of  expertise  may 
be  useful  for  the  measurement  and  evaluation  of  nation  building  activities. 

Though  defense  organizations  have  underutilized  such  frameworks,  other  international  organizations 
have  a  history  of  employing  them.  An  early  attempt  at  measuring  progress  in  nation  building  can  be  found 
in  a  1997  DFI  International  study  on  effective  transitions  in  United  Nations  peace  operations  (Blechman 
et  al.,  1997).  Another  example  is  the  World  Bank  Low-Income  Countries  Under  Stress  (LICUS) 

Initiative’s  transitional  results  matrix  that  has  sought  to  measure  progress  in  a  variety  of  countries, 
including  Liberia  and  Haiti.  The  Fund  for  Peace  developed  a  comprehensive  model  that  was  applied  to 
Iraq.  Furthermore,  the  Center  for  Strategic  and  International  Studies  developed  a  model  for  measuring 
progress  and  applied  it  to  both  Iraq  and  Afghanistan’s  reconstruction  (Cohen,  2006). 

More  recently  however,  military  organizations  -  including  NATO,  which  is  leading  the  International 
Security  Assistance  Force  (ISAF)  mission  in  Afghanistan  -  have  begun  to  recognize  the  importance  of 
management  accounting,  and  have  implemented  their  own  effect  measurement  methodologies.  ISAF’s 
methodology  is  known  as  an  effects-based  approach  to  operations  (EBAO).  In  general,  EBAO  seeks 
to  create  a  holistic  picture  of  the  operational  environment  to  enhance  military  planning,  the  conduct  of 
operations,  and  the  assessment  of  the  efficacy  of  those  operations  (Prescott,  2008).  Within  this  broader 
view,  commanders  and  staff  at  all  levels  should  then  be  able  to  synchronize  their  efforts  with  those  of 
other  governmental,  nongovernmental,  and  international  organizations. 

Measuring  the  progress  and  the  performance  of  organizations  that  participate  in  nation  building  and 
reconstruction  is  important  for  several  reasons.  First,  such  measurements  are  likely  to  create  transparency 
and  can  thus  contribute  to  increased  donor  and  governmental  accountability  (Noordegraaf  and  Abma, 
2003;  Thiel  and  Leeuw,  2002;  de  Bruijn,  2007).  Through  undertaking  such  measurements,  individuals 
and  organizations  can  be  cognizant  of  the  exact  number  of  resources  used  in  any  given  project  (Glenn 
and  Gayton,  2008).  Second,  such  measurements  enable  the  evaluation  of  outputs  and,  therefore,  the 
strengthening  of  effective  administration  (Noordegraaf  and  Abma,  2003).  Based  on  this  evaluation, 
military  commanders  and  civilian  leaders  can  reallocate  resources  or  adjust  their  strategic  planning  as 
necessary.  Third,  such  progress  measurement  enables  organizations  to  learn  what  they  do  well  and  when 
improvements  are  possible  (de  Bruijn,  2007).  A  final  reason  is  that  such  measurement  can  improve  the 
communication  between  participating  organizations  such  as  NATO,  the  United  Nations  and  host  nation 
authorities.  This  can  help  align  the  international  community’s  expectations  with  those  of  the  host  nation 
stakeholders,  a  phenomenon  that  United  States  General  David  Petraeus  termed  ‘managing  expectations’ 
(Petraeus,  2008). 

However,  applying  accounting  and  control  concepts  and  measurements  to  reconstruction  activities 
is  not  without  difficulties.  First,  a  retrospective  look  at  efforts  to  establish  measures  of  progress  in 
nation  building  indicate  that  operations  between  organizations  have  been  disconnected  from  each  other, 
characterized  by  a  range  of  disparate  terminologies  and  methodologies  (Cohen,  2006).  Further,  with 
the  presence  of  so  many  metrics  within  many  institutions  (Thiel  and  Leeuw,  2008),  selecting  the  right 
measures  is  difficult.  Organizations  must  increasingly  balance  the  desire  to  maintain  simple,  easily 
assessed,  comprehensible  metrics  with  the  so-called  “kitchen-sink  approach,”  in  which  increased  data 
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collection  and  subsequent  analyses  attempt  to  satisfy  all  prospective  users’  requirements  (Glenn  and 
Gayton,  2008). 

Second,  finding  a  causal  relationship  between  actions  and  outcomes  is  difficult  in  general,  but 
particularly  so  in  nation  building  and  reconstruction.  To  establish  causality  -  that  action  A  results  in 
outcome  B  -  requires  that  very  specific  and  oftentimes  impossible  conditions  be  met  (Glenn  and  Gayton, 
2008).  A  commander  of  an  American  military  unit  observed: 

Even  if  we  can  successfully  measure  an  outcome,  it’s  extremely  hard  to  know  what  caused 
the  outcome.  There  are  so  many  things  happening  at  once  that  causal  relationships  are  next  to 
impossible  to  identify.  There  is  a  certain  amount  of  guessing  and  operational  art  in  measuring 
success  (Glenn  and  Gayton,  2008). 

Third,  the  process  of  measurement  frequently  increases  bureaucracy.  When  an  organization  emphasizes 
performance  measurement,  it  often  assigns  considerable  resources  to  producing  data  on  results  and 
impact.  This  can  increase  the  load  of  bureaucracy  enormously.  Power  (1994)  even  refers  to  this 
phenomenon  as  the  “audit  explosion”  or  “audit  society.” 

Finally,  the  complex  relationship  between  civilian  agencies  and  military  actors  can  hamper  an  integrated 
attempt  to  measure  the  performance  of  nation  building.  Although  many  researchers  and  practitioners 
support  the  idea  that  successfully  coordinated  civilian  and  military  efforts  are  a  key  to  successful  nation 
building  (Rietjens,  2008),  civilian  agencies’  associations  with  a  potentially  unwelcome  military  force  is 
risky  and  can  result  in  the  agencies  losing  the  protective  patina  of  neutrality  (Donini  et  al.,2004;  Macrae, 
2002;  Wheeler  and  Harmer,  2006;  Hasegawa,  2008).  Since  civilian  actors  and  their  military  counterparts 
frequently  have  different  objectives  and  different  ways  of  achieving  these  (Rietjens,  2008),  they  look 
favorably  upon  cooperation  as  long  as  they  expect  it  to  serve  their  best  interests  (Seiple,  1996).  This 
can  easily  lead  to  opportunistic  behavior.  Moreover,  the  differences  in  organizational  culture,  expertise, 
resources  and  timeframes  between  the  two  sets  of  actors  also  contribute  to  this  complexity  (Abiew,  2003; 
Bollen,  2002). 

Afghanistan  is  an  exemplary  case  of  this  last  problem,  a  country  in  which  many  military  and  civilian 
organizations  operate  side  by  side  in  the  process  of  nation  building,  and  thus  tandemly  attempt  to  measure 
their  activities’  performances.  The  International  Security  Assistance  Force  (ISAF)  uses  EBAO,  while 
USAID,  the  United  Nations  Mission  of  Afghanistan  (UNAMA)  and  other  civilian  agencies  use  their 
own  control  systems.  These  inconsistencies,  coupled  with  the  aforementioned  problems  inherent  to 
performance  measurement,  result  in  incorrect,  incomplete  and  politically  motivated  presentations  of  the 
performance  regarding  various  aspects  of  nation  building  (Rietjens,  2008;  Glenn  and  Gayton,  2008). 

3.  Methodology 

Elaving  recounted  the  definition,  history,  benefits  and  drawbacks  of  assessment  monitoring  in  nation 
building,  the  discussion  now  turns  to  the  case  study  at  hand.  As  mentioned  in  the  introduction,  this  article 
uses  the  ACSP  to  develop  new  insights  and  understandings  of  nation  building  activities  in  Afghanistan. 
This  85,000-project  database,  created  specifically  to  manage  and  control  progress  and  performance 
during  nation  building,  reconstruction,  and  development  activities  in  Afghanistan,  is  sourced  from  a 
multiplicity  of  entities  including  the  Afghan  government,  donors,  provincial  reconstruction  teams  and 
non-governmental  organizations  (NGOs).  It’s  collective  information  covers  a  range  of  efforts,  from  the 
reconstruction  of  roads,  bridges,  dams  and  schools  to  the  development  and  introduction  of  basic  health 
packages  in  the  country.  The  ACSP’s  primary  purpose  is  to  enable  widespread  situational  awareness  and 
coordination  throughout  the  Afghan  government,  NATO  and  the  development  community  and  to  align 
and  assess  their  strategic  programs  in  Afghanistan.  Because  of  the  varied  sources  of  its  information,  it 
cannot  be  controlled  by  one  organization  (e.g.,  ISAF)  but  -  in  theory  -  can  be  used  to  synchronize  efforts 
between  organizations. 
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The  first  part  of  our  research  consisted  of  interviews,  briefings  and  participation  in  meetings  with  NATO 
officials  during  the  July-November  2007  deployment  of  our  first  author  to  Kandahar.  This  deployment 
enabled  him  to  develop  a  more  profound  understanding  of  the  background,  architecture  and  relevance  of 
the  ACSP.  In  2008,  using  relevant  literature,  we  further  analyzed  and  discussed  the  resulting  dataset  from 
this  period.  Subsequently,  we  re-coded  the  initial  dataset  for  statistical  testing  and  examined  the  collected 
data. 

In  January  2009,  our  remaining  two  authors  conducted  a  second  part  of  the  research  during  a  field  visit 
to  Afghanistan.  As  with  the  first  portion  of  our  study,  they  met  with  different  NATO  officials  in  Kandahar 
as  well  as  Kabul.  This  second  part  proved  useful  in  refining  our  understanding  of  the  ACSP  and  updating 
our  own  ACSP  dataset.  Moreover,  this  period  allowed  us  to  conduct  interviews  in  much  more  detail  and 
discuss  our  earlier  findings  with  practitioners. 

3.1.  Data 

The  ACSP  dataset,  which  served  as  input  for  our  research,  consisted  of  84,641  project  records.  The 
original  dataset  used  25  variables  including  identification  and  information  processing  characteristics  that 
were  less  relevant  for  our  purpose.  After  analyzing  the  dataset  in  combination  with  interviews,  briefings, 
meeting  reports  and  literature,  we  decided  to  use  a  number  of  these  25  variables  as  our  basic  set. 1  Among 
these  were:  “start  date,”  “end  date,”  “region,”  “cost,”  “status,”  “Afghan  national  development  strategy 
(ANDS)  sector,”  and  “implementing  partner.”  In  addition,  we  defined  two  new  variables:  “days  completed 
per  project”  and  “turnover  per  project  per  day”  to  address  the  absorption  capacity  in  Afghanistan. 

As  a  second  measure,  we  re-coded  the  variable  “region”  to  a  new  variable  named  “security  situation” 
using  polling  data  we  collected  during  our  field  visits.  From  2006  to  2008,  field  workers  conducted  eight 
polls  (N  =  5650)  throughout  Afghanistan  to  develop  insight  into  the  perceptions  of  Afghans  regarding 
security.2  We  used  the  polling  question,  “How  is  the  security  situation  in  your  area?”,  allowing  the 
participant  to  provide  a  rating  between  one  (worst)  and  ten  (best).  The  question  and  its  results  facilitated 
the  classification  of  the  five  NATO  regions:  the  capital  (Kabul  and  surroundings),  the  north,  the  east, 
the  west  and  the  south.  An  examination  of  the  polling  data  showed  one  region  with  a  mean  of  five,  two 
regions  with  a  mean  of  seven  and  two  regions  with  a  mean  of  eight.  We  cross  referenced  the  polling  date 
with  Gauster  (2008)  and  Asia  Foundation  (2007)  and  coded  the  regions  in  three  security  classes:  insecure 
(south),  medium  secure  (east  and  west)  and  relatively  secure  (north  and  capital). 

Third,  we  re-coded  the  variable  “project  status”  to  the  indicators  “completed,”  “ongoing,”  “cancelled/ 
suspended,”  and  “unknown.”  In  this  way,  time-related  analyses  were  possible  and  differences  between 
regions  could  be  analyzed. 

Fourth,  we  created  a  dichotomous  variable  to  indicate  local  participation.  We  therefore  re-coded  the 
variable  “implementation  partner”  and  used  the  involvement  of  the  Afghan  community  as  the  indicator  for 
local  participation. 

Our  fifth  measure  was  the  creation  of  dummy  variables  to  conduct  a  regression  analysis  with  the 
“security  situation,”  “Afghan  participation,”  and  “military  (CIMIC)  versus  non-military  (USAID).”  We 
subsequently  validated  the  ACSP  variables  with  graphs  and  descriptive  analyses  to  identify  outliers  and 
cells  that  contained  erroneous  data  (such  as  start  dates  before  2002  or  after  2009,  and  days  completed 
less  than  zero).  The  ACSP  database  holds  data  that  is  updated  at  different  intervals.  To  eliminate  possible 
effects  of  data  that  has  not  been  updated  near  the  end  of  2008,  and  to  eliminate  projects  that  were  not 
monitored  after  initial  registration,  we  did  not  use  projects  with  missing  cells  on  “start  date”  and  projects 
started  after  2007.  On  the  basis  of  these  examinations  we  discarded  23,418  records,  which  decreased  our 
final  dataset  to  61,223  projects  ranging  in  date  from  2002  to  2007. 
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4.  Findings 

Projects  and  Strategies 

Table  1  indicates  the  regions  in  which  various  projects  have  been  carried,  between  2002  and  2007.  Of 
the  61,223  projects  in  the  database,  over  44%  have  been  executed  in  the  capital  and  the  eastern  part  of 
the  country.  This  is  not  surprising  since  the  activities  of  the  international  community,  both  military  and 
civilian,  have  typically  first  originated  in  Kabul.  A  second  reason  for  the  heavy  concentration  of  projects 
in  the  north  and  east  is  that  these  regions  have  the  highest  population  densities  in  the  country,  and  thus  the 
need  for  support  tends  to  be  the  highest.  However,  in  2004,  military  and  development  activities  started  to 
incorporate  other  regions  of  the  country,  expanding  from  their  initial,  relatively  safe  northern  and  eastern 
locations  to  include  the  relatively  insecure  southern  areas. 

Further  analyses  indicate  that  very  few  projects  have  been  undertaken  in  the  regions  immediately 
bordering  Pakistan,  most  notably  along  the  so-called  Federally  Administered  Tribal  Area.  In  these  regions 
conflict  is  rife  (e.g.  Rashid,  2008;  Johnson  and  Mason,  2008),  thus  preventing  safe  working  conditions  for 
military  personnel  and  development  agents  alike.  Moreover,  the  relatively  paltry  number  of  projects  in 
this  region  can  be  explained  by  the  region’s  low  population  density,  the  limited  number  of  ISAF  troops, 
and  the  significant  distance  from  the  nation’s  political  center.  These  developments  highlight  the  fact  that 
so-called  “structural  holes”  in  project  implementation  may  occur  due  to  understandable  lopsided  attention 
of  international  and  domestic  actors  to  the  ‘easier’  operational  environments  in  the  country  (Burt,  1992). 

Table  1  shows  that  more  than  61,000  projects  have  been  executed  in  the  period  between  2002  and  2007. 
With  some  200  projects  in  2002,  the  subsequent  year  saw  projects  rise  60  times  to  a  total  of  12,318  by 
2004.  Hence,  in  the  first  three  years,  there  was  a  steep  rise  in  the  number  of  projects,  with  the  number 
remaining  stable  in  2004,  2005  and  2006.  The  next  year,  in  2007,  projects  rocketed  up  to  almost  22,000. 

Table  1:  Projects  in  Different  Regions  by  Year 


StartYear 

2002 

2003 

2004 

2005 

2006 

2007 

Total 

Region 

Capital 

22,6% 

6,5% 

4,5% 

6,0% 

4,8% 

5,4% 

5,3% 

East 

36,0% 

48,0% 

42,7% 

46,5% 

45,1% 

28,5% 

39,1% 

North 

8,8% 

24,9% 

20,2% 

25,0% 

18,7% 

23,9% 

22,3% 

South 

22,6% 

12,2% 

23,6% 

9,1% 

10,2% 

30,0% 

19,8% 

West 

9,6% 

7,4% 

8,7% 

12,6% 

20,7% 

11,1% 

12,7% 

Unknown 

0,4% 

1,0% 

0,3% 

0,8% 

0,6% 

1,1% 

0,8% 

Total 

239 

2232 

12318 

12279 

12481 

21674 

61223 

In  connection  to  Table  1,  Figure  1  shows  the  various  types  of  projects  undertaken  in  Afghanistan. 
Information  about  the  mix  of  projects  and  developments,  over  time,  can  be  important  to  understanding 
whether  early  strategies  -  such  as  those  intended  to  have  quick  impacts  -  eventually  transformed  into, 
for  example,  long-term  projects  supporting  the  Afghan  government.  Figure  1  indicates  that  the  majority 
of  projects  (almost  45%)  relate  to  the  development  of  infrastructure  and  natural  resources  (roads,  water 
supply,  power).  The  second  greatest  number  are  agricultural  and  rural  development  projects  (20%), 
followed  by  projects  focused  on  education,  social  protection  and  health.  Projects  in  the  field  of  good 
governance  are  still  relatively  scarce,  as  are  those  relating  to  private  sector  development.  The  former 
condition  might  be  explained  by  the  heavily  entrenched  and  rather  rigid  nature  of  Afghan  politics,  while 
the  latter  might  be  explained  by  the  fact  that  military  personnel  and  aid  workers  are  rarely  hired  to 
undertake  wide-reaching  macro-economic  reforms.  This  indicates  that  the  institutional  conditions  for 
Western-style  economic  development  in  the  country  are  still  weak. 
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Figure  1:  Projects  and  the  Afghan  National  Development  Sector  Strategy > 


ANDS  Sector 


Organizations  follow  divergent  strategies  to  achieve  their  goals.  Because  USAID  and  the  military’s 
CIMIC  framework  are  the  largest  project  undertakers  in  Afghanistan  (accounting  for  some  20,000  projects 
altogether),  we  primarily  analyze  differences  in  the  projects’  foci  for  these  two  entities.  Table  2  offers  a 
breakdown  of  the  projects  into  activities  conducted.  USAID  is  particularly  inclined  to  conduct  projects 
in  the  fields  of  agriculture  and  rural  development  (about  40%  of  its  total  projects)  and  education  (17%), 
whereas  ISAF  focuses  not  only  on  infrastructure  and  natural  resources  (27%  of  their  projects),  but  also  on 
social  protection  (almost  18%)  and  education  (17%).  The  relatively  few  projects  relating  to  private  sector 
development  are  initiated  five  times  more  often  by  USAID  than  by  the  military.  All  these  differences  are 
significant  (Chi-Square-test,  p=  0,00).  The  focus  of  projects  conducted  by  ISAF  may  be  explained  by  the 
military  ‘can-do’  culture  as  well  as  its  engineering  orientation  toward  quick  fixes  and  tangible  results. 

This  may  also  stem  from  the  relatively  short  stay  of  military  personnel  in  the  mission  area.  Infrastructure, 
natural  resource  development  (in  particular  energy)  and  to  a  lesser  extent  education  are  other  examples  of 
such  quick  impact  projects. 

Table  2:  CIMIC  and  USAID  Projects 


ANDS  Sector 

CIMIC 

USAID 

Total 

Agriculture  &  Rural  Development 

7,1% 

40,7% 

30,4% 

Education 

17,4% 

17,3% 

17,4% 

Good  Governance 

9,7% 

9,0% 

9,2% 

Health 

10,1% 

8,6% 

9,1% 

Infrastructure  Sc  Natural 

Resources 

27,7% 

9,7% 

15,2% 

Private  Sector  Development 

1,3% 

6,6% 

5,0% 

Security 

6,9% 

0,5% 

2,4% 

Social  Protection 

18,3% 

7,6% 

10,8% 

Unknown 

1,5% 

0,0% 

0,5% 

Total 

6114 

13869 

19983 

4.1.  Absorption  Capacity 

The  absorption  capacity,  defined  as  the  average  expenditure  or  turnover  per  project  per  day,  can  be  used 
as  one  of  the  indicators  to  measure  progress  in  an  area  of  operation  or  of  an  organization.  Between  2002 
and  2005,  throughout  Afghanistan’s  various  regions,  the  absorption  capacity  remained  nearly  stable.  In 
2006,  it  rose  rapidly  up  to  $2,500.  However,  in  the  southern  region  and  in  the  capital  of  Kabul,  the  year 
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2007  showed  a  steep  decline  in  mean  project  turnover  costs.  This  fall  was  no  doubt  precipitated  by  the 
dramatically  deteriorating  security  situation  in  those  areas  beginning  in  2006,  (Rashid,  2008),  a  reality 
that  was  underlined  by  the  results  of  our  own  polling  amongst  local  populations  in  the  various  regions 
of  the  country.3  During  this  same  period,  expenditures  in  the  western,  northern  and  especially  eastern 
regions  stabilized  or  even  grew. 

Figure  2  shows  that  the  eastern  and  the  western  regions  -  both  of  which  are  relatively  safe  -  have 
the  highest  mean  expenditure  per  project  per  day,  whereas  the  more  insecure  south  has  the  lowest.  In 
Kabul,  and  in  the  northern  part  of  the  country,  the  mean  expenditure  is  twice  as  high  as  in  the  south  but 
significantly  lower  than  in  the  east  and  the  west.  The  non-parametric  tests  indicate  significant  differences 
between  the  ranks’  mean  turnover  per  day  (Chi  Square-test;  p=0,00). 

Figure  2:  Mean  Turnover  per  Day  and  Security  Situation 
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Security  Situation 

4.2.  Afghan  Participation 

An  important  aspect  of  reconstruction  activities  in  conflict-ridden  countries  such  as  Afghanistan  is 
that  domestic  civil  society  must  view  the  international  actors  within  the  country  as  being  welcomed 
and  legitimate.  This  legitimacy  is  likely  to  grow  when  local  partners  participate  in  the  formulation, 
development  and  implementation  of  the  projects,  as  it  helps  to  encourage  sustainability,  local  ownership 
and  capacity  building  (Natsios,  2005;  Narten,  2008;  Rietjens  et  al.,  2009).  Developments  and  information 
regarding  the  participation  of  local  partners  is  therefore  an  important  aspect  in  measuring  progress  in 
nation  building  and  the  performance  of  organizations. 

We  distinguished  between  projects  that  were  the  sole  responsibility  of  international  actors  (line  “0” 
in  Figure  3)  and  projects  that  were  conducted  largely  by  local  Afghan  institutions  and  agencies  (line 
“1”  in  Figure  3).  Figure  3  demonstrates  that  during  2002,  Afghan  stakeholders  participated  in  very  few 
projects.  In  2003,  Afghan  participation  increased  and  in  2004,  projects  undertaken  in  tandem  with  Afghan 
stakeholders  even  outnumbered  those  undertaken  without  local  input.  The  decrease  in  local  participation 
in  2006  is  remarkable  and  comprehensible  only  by  pointing  again  to  the  decline  in  the  security  situation. 

Several  reasons  may  be  identified  as  to  why  many  projects  have  been  conducted  without  the 
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involvement  of  Afghan  stakeholders.  While  the  central  government  holds  extensive  constitutional 
authority  over  the  provinces,  Kabul’s  limited  de  facto  ability  to  intervene  and  its  consequent 
accommodation  of  small-scale  power  brokers  in  the  provinces  have  left  factional  chiefs  in  control  of 
many  local  governments  (Jalali  2006).  This  often  makes  it  very  difficult  for  international  actors  -  both 
military  and  civilian  -  to  identify  reliable  and  legitimate  Afghan  partners  for  the  executed  projects. 
Another  reason  for  the  sometimes  limited  involvement  of  Afghan  stakeholders  may  be  the  negative 
attitudes  that  Afghan  civilian  actors  hold  towards  foreigners,  most  notably  Western  military  members. 
Examples  abound  of  local  villagers  being  threatened,  injured  or  even  killed,  after  interacting  with  foreign 
troops.  These  dangerous  precedents  clearly  have  a  deleterious  impact  on  the  willingness  of  local  Afghans 
to  participate  in  ISAF  and  USAID  projects.  A  final  reason  likely  relates  to  timing.  Civilian  organizations, 
especially  development  organizations,  are  sometimes  prepared  to  stay  in  a  given  country  for  a  period  of 
five  to  ten  years  to  execute  their  projects,  whereas  the  military  will  often  have  a  time  horizon  limited  to 
two  or  three  years.  Most  ISAF  units  rotate  their  personnel  every  four  to  six  months,  and  this  high  turnover 
rate  likely  hampers  their  interactions  with  Afghan  stakeholders. 

Figure  3:  Local  Participation 


Start  Y  ear 

Table  3  shows  that  the  distribution  of  Afghan  participation  in  the  various  regions  differs  significantly 
(Chi  Square  test;  p=0,00).  In  particular,  the  southern  region  contains  relatively  few  projects  in  which 
Afghan  stakeholders  were  involved.  In  contrast,  the  situation  is  opposite  for  the  northern  region. 

These  differences  can  be  explained  by  the  problematic  security  situation  in  the  southern  region,  which 
hampers  both  the  military  and  civilian  agencies  to  freely  move  through  the  area  and  consult  with  Afghan 
stakeholders  on  a  frequent  basis.  Moreover,  as  mentioned  previously,  Afghan  participants  in  the  southern 
districts  may  feel  more  reluctant  to  participate  in  projects  initiated  by  the  international  community. 

Table  3:  Local  Participation  and  Regions 


Afghan  Participation 

Region 

No 

Yes 

Total 

Capital 

3,2% 

2,1% 

5,3% 

East 

17,0% 

22,1% 

39,1% 

North 

7,3% 

15,0% 

22,3% 

South 

15,1% 

4,8% 

19,9% 

West 

5,9% 

6,8% 

12,7% 

Unknown 

0,7% 

0,0% 

0,7% 

Total 

30217 

31006 

61223 
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4.3.  Experience  and  Learning 

In  addition,  we  have  tried  to  reveal  the  completion  time  for  projects.  This  is  useful  information,  as 
organizations  tend  to  take  a  longer  time  to  complete  projects  upon  initial  entry  into  a  country  -  when 
they  still  need  to  assess  local  communities  -  than  after  months  of  in-country  experience.  Presumably, 
the  longer  time  that  an  organization  spends  operating  in  a  specific  post-conflict  environment,  the  better  it 
becomes  at  navigating  local  terrain  and  the  quicker  it  can  complete  projects. 

Figure  4  shows  interesting  results  in  this  regard.  We  have  calculated  the  average  number  of  days  to 
project  completion  in  various  years.  The  figure  shows  that  between  2003  and  2004,  the  duration  of 
projects  increased  gradually.  This  figure  declined  sharply  in  the  following  two  years.  After  stabilizing  in 
2006,  it  declined  again  in  2007. 

Figure  4:  Experience  Curve 


Start  Year 


The  graph  above  clearly  resembles  the  so-called  “experience”  curve,  popular  in  economic  theory. 

This  curve  indicates  systematic  reductions  in  production  costs  and  time  over  the  life  of  a  product.  Two 
phenomena  explain  this:  learning  effects  and  economies  of  scale  (e.g.  Hill,  2001).  Learning  effects  refer 
to  cost  and  time  savings  that  come  from  learning  by  doing  and  repeating  the  same  activities.  Labor 
productivity  increases  over  time  as  individuals  learn  the  most  efficient  ways  to  perform  their  tasks. 

We  can  see  both  effects  quite  clearly  in  the  projects  in  Afghanistan.  In  the  beginning  of  the  peace 
support  and  reconstruction  operations  in  early  2002,  international  organizations  needed  to  learn  how  to 
operate  in  the  new,  foreign  environment.  Apparently,  after  this  start-up  stage  they  became  more  adept 
at  dealing  with  local  participants,  and  correspondingly,  there  is  a  rapid  decrease  in  the  average  project 
completion  time.  Stabilization  of  this  dynamic  in  2006  likely  reflects  military  and  NGO  adjustments  to 
increased  hostilities.  In  2007,  the  average  project  completion  time  decreased  again.  This  is  an  important 
finding,  as  success  and  speed  generate  confidence  in  the  reconstruction  process.  Indeed,  this  is  so 
important  that  we  wanted  to  test  the  effect  in  a  robust  multivariate  model. 

In  this  analysis,  the  time-related  “experience”  effect  turned  out  to  be  the  most  important  factor  in 
explaining  the  variance  in  project  durations  (“days  completed”).  To  avoid  artificially  high  correlations,  we 
conducted  this  analysis  on  the  projects  that  commenced  before  2007  and  did  not  last  longer  than  365  days; 
hence,  the  relatively  shorter  projects  (about  7,000).  We  conducted  this  analysis  using  “start  date”  (beta  = 

-,  37,  sign.),  a  dummy  variable  concerning  the  security  situation  in  the  country  (beta  =  -,  03,  sign.),  the 
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“Afghan  participation”  (beta  =  0,003,  non-sign.),  and  a  dummy  variable  “CIMIC-USAID”  (beta=  -,  13, 
sign.)  as  independent  variables.  This  analysis  (adj.  R  square  is  0,197)  indicated  that  the  duration  of  this  set 
of  relatively  shorter  projects  (average  142  days)  is  very  strongly  influenced  by  -  as  said  -  the  net  effect 
of  the  start  date  (the  more  recent,  the  shorter),  slightly  by  the  security  situation  (shorter  in  safer  regions), 
and  somewhat  strongly  by  who  has  initiated  the  project  (the  military  projects  focusing  on  infrastructure 
took  on  average  almost  30  days  less  to  be  completed  than  the  USAID  projects).  Afghan  participation  does 
not  play  a  significant  role  in  this  analysis.  This  regression  analysis  shows  the  strength  of  the  “experience” 
curve  over  the  other  significant  variables,  i.e.  the  safety  situation  and  the  type  of  projects  (as  indicated  by 
the  difference  between  military  and  USAID  projects). 

5.  Discussion  and  Conclusion 

Over  the  last  several  decades,  performance  measurement  has  become  increasingly  important  in  public 
sector  administration.  Politicians  and  civil  societies  alike  want  to  know  about  the  results  achieved  with  the 
resources  expended  by  the  government  on  their  behalf  (de  Bruijn,  2007);  more  specifically,  they  want  to 
know  about  the  benefits  generated  for  society  at  large  in  the  expenditure  of  these  resources. 

So  far,  however,  this  development  has  not  been  very  strong  in  the  field  of  foreign  development 
assistance  and  aid  policies.  The  cry  for  help  from  disaster-struck  and  conflict-ridden  regions  in  the  world 
is  often  so  emotionally  charged  that  asking  for  insight  into  efficiency  and  effectiveness  seems  to  be 
almost  inhuman.  Thanks  to  this  and  various  other  factors,  we  know  surprisingly  little  about  where  aid  and 
assistance  money  flows,  and  how  well  it  is  actually  spent  (e.g.  Collier,  2007). 

The  same  applies  to  military  expenditures  in  post-conflict  societies.  However,  given  that 
developmental  assistance  and  current  military  operations  are  policies  of  choice  in  regions  of  choice, 
these  activities  will  increasingly  have  to  compete  with  policy  expenditures  at  home,  such  as  improving 
the  national  economy,  national  education  and  national  health  care.  Therefore,  governments  need  to 
demonstrate  the  value  of  the  activities  that  are  accomplished  in  those  far-flung  regions  to  their  domestic 
constituencies.  Proving  the  value  of  such  activities  is  difficult.  As  we  have  seen  previously,  in  the  public 
sector,  measuring  progress  in  the  delivery  of  policies  is  complicated.  There  are  more  stakeholders  in  the 
political  arena  who  have  their  own  views,  perceptions  and  ambitions,  all  of  which  are  less  tangible  and 
confusingly  interrelated. 

All  of  this  applies  equally  to  nation  building  and  reconstruction  in  post-conflict  areas,  such  as 
Afghanistan.  Yet,  it  is  inevitable  in  such  endeavors  that  some  sort  of  progress  and  performance 
measurement  will  be  used  in  order  to  provide  at  least  an  indication  of  what  is  being  reached.  In  this 
article  we  have  used  a  database  consisting  of  quantitative  data  concerning  the  projects  that  have  been 
implemented  in  the  framework  of  nation  building  in  Afghanistan  in  the  period  2002-2008. 

We  have  been  able  to  show  that  there  is  a  gradual  but  strong  growth  in  the  number  of  projects 
over  the  years;  that  projects  are  increasingly  more  evenly  spread  over  the  regions  in  the  country; 
that  a  threatened  security  situation  plays  a  hampering  role;  that  Afghan  participation  in  the 
projects  is  not  small,  yet  the  effectiveness  of  this  legitimacy  is  largely  unknown;  that  the  projects 
are  executed  faster  over  the  years,  commensurate  with  the  nation  builders’  experience;  and  that 
the  military  and  the  civil  assistance  workers  seem  to  have  developed  a  division  of  labor  since 
they  focus  on  different  types  of  projects. 

This  seems  to  be  good  news.  However,  the  ACSP  database  contains  no  information  on  the 
real  value  that  the  completed  projects  constitute  in  the  eyes  of  the  local  population,  and  no 
information  on  how  the  results  of  the  projects  are  being  used,  and  by  whom.  For  these  reasons, 
it  is  important  to  expand  the  data  with  a  number  of  other  characteristics.  However,  it  is  first 
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necessary  to  share  the  results  of  this  data  with  the  local  and  regional  stakeholders  and  to 
determine  the  completed  projects’  value  to  the  local  communities.  Too  often,  such  information 
stays  in  the  upper  echelons  of  the  international  civilian  and  military  organizations,  precluding 
profitable  discussions  with  local  political  entities  (Ghani  and  Lockhart,  2008).  In  nation  building, 
enhancing  the  ‘local  footprint’  appears  necessary  in  all  stages  of  policy  making,  not  the  least  in 
progress  and  performance  measurement. 
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ANNEX  A  ACSP  VARIABLES 


Number 

Variable 

Description 

Used  in 
Research 

Database 

1 

ID 

Identification 

Number 

N 

2 

HQ_1SAF  URN 

Identification 

Number 

N 

3 

PROJECT  NUMBER 

Identification 

Number 

N 

4 

REGION 

ISAF  Region 

Y 

5 

PRT 

Provincial 

Recontruction 

Team 

N 

6 

ANDS  SECTOR 

Afghan  National 

Development 

Strategy 

Y 

7 

ASIC  SECTOR 

Afghanistan 

Standard  Industrial 
Classification 

N 

8 

ASIC  ACTIVITY 

Afghanistan 

Standard  Industrial 
Classification 

Activity 

N 

9 

CURRENT 

ACTIVITY 

DESCRIPTION 

Description  of 
activities 

N 

10 

PROVINCE 

Afghan  Province 

Y 

11 

DISTRICT  NAME  2 

Specification  in 
Province 

N 

12 

VILLAGE 

Village  Name 

N 

13 

LATITUDE 

Latittude 

Y— only 
plots  during 
research 

14 

LONGTITUDE 

Longitude 

Y— only 
plots  during 
research 

15 

LOC  MGRS 

Military  Grid 

Y— only 

18 


Reference  System 
locator 

plots  during 
research 

16 

COST 

Costs  in  US  Dollar 

Y 

17 

MINISTRY  ABBR 

Abbreviation  of 
Ministry  Policy 

Area 

N 

18 

IMPLEMENTING 

PARTNER 

Partner  of  ISAF 
for  implementation 
of  a  project 

Y 

19 

STATUS 

Status  of  project 

Y 

20 

START  DATE 

Start  date  of 
project 

Y 

21 

END  DATE 

End  date  of  project 

Y 

22 

DONOR 

Donor  for  project 

N 

23 

INFO  DATE 

Date  records  are 
updated 

N 

24 

SOURCE 

Information  Source 

N 

25 

REMARKS 

Remarks 

N 

(Endnotes) 


1 .  The  ACSP  database  consists  of  four  main  data  groups  that  are  combined  in  one  database:  CIMIC  (i.e.  the  projects  of  ISAF  units),  the  projects  of  the  Afghan  Ministry  of  Rural  Rehabilita¬ 
tion  and  Development  (MRRD),  USAID  and  miscellaneous.  The  individual  project  records  are  updated  using  25  variables.  These  variables  are  specified  in  annex  A.  To  focus  this  research 
and  point  our  analyses  we  screened  all  25  variables  on  usefulness  and  uniqueness.  For  example,  strategy  measurement  and  regional  differences  are  notable  for  performance  measurement 
of  nation  building.  Differences  between  the  financial  support  of  donor  countries  are  for  this  research  less  useful.  Since  the  dataset  is  large,  approximately  85,000  project  records,  the 
variables  were  screened  rigorously  to  give  way  to  clear  analyses.  Calculation  and  re-coding  of  variables  were  then  necessary  steps  to  further  detail  our  research  results.  For  example  the 
variables  “cost”,  “start  date”  and  “end  date”  have  been  used  to  calculate  the  variable  “mean  turnover  per  day”  to  analyze  the  absorption  capacity  as  a  performance  measurement  indicator. 

2.  MRA  Institute  &  ISAF  (2008).  ANQAR  1 .0:  Afghanistan  Nationwide  Quarterly  Assessment  Research.  Final  presentation  COMISAF.  November  8,  2008. 

3.  MRA  Institute  &  ISAF  (2008).  ANQAR  1 .0:  Afghanistan  Nationwide  Quarterly  Assessment  Research.  Final  presentation  COMISAF.  November  8,  2008 
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